MDEV-39509: Fix "invalid iv length" with OpenSSL 4.0
OpenSSL 4.0 changed the behavior of EVP_CIPHER_CTX_new:
It now initializes the internal iv_len with a sentinel
value of -1. When calling EVP_CipherInit_ex with the
cipher, key, and IV simultaneously, the provider validates
the IV against this sentinel before the cipher's default
metadata is applied, triggering an error.
This patch implements a two-stage initialization for OpenSSL 4.0+:
- Call EVP_CipherInit_ex with only the cipher to "thaw" the
context and set the correct default iv_len.
- Perform a second call to provide the actual Key and IV data.
This approach ensures compatibility with OpenSSL 4.0 while
avoiding EVP_CipherInit_ex2 to maintain support for FIPS mode
and older versions.
(cherry picked from commit
5ca4ab639ad5f99d74f794b54ba1b4d6182ec2a9 from
upstream feature branch, not yet merged)
Forwarded: https://github.com/MariaDB/server/pull/5029
Gbp-Pq: Name MDEV-39509-Fix-invalid-iv-length-with-OpenSSL-4.0.patch